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Impact of Instructional Supervision on Student Achievement: Can We Make the 

Connection? 



Abstract 



This paper reports on the final phase of a three-part study on the status of instructional 
supervision within several New York City public schools. In the first parts of the study 
the researchers found, through extensive use of surveys (questionnaires and interviews), 
that centralized educational reform had serious consequences for instructional 
supervision. Results indicated that in many instances principals, given many non- 
instructional duties, did not have the time to undertake continuous and meaningful 
supervision. Often, such supervision was relegated to coaches, neither trained in 
supervision nor given organizational authority to effectuate needed reforms to ensure 
quality teaching. Teachers in many cases indicated that supervision was perfunctory and 
evaluative. The researchers concluded that the highly centralized system of schooling 
that mandated prescribed curricula, added responsibilities for supervisors, and instituted 
narrow definitions of accountability aimed to, above all else, hold principals (and thus 
teachers) accountable for increases in student achievement transformed instructional 
supervision into a monitoring function, at its best. Several instances of effective models 
of supervision and professional development were discovered, however, despite 
bureaucratic and other non-school related constraints. These schools have had significant 
increases in student achievement levels as reported by State standardized tests. This last 
paper in the series summarizes some findings from one such successful school utilizing in 
depth methodologies aimed to uncover the relationship between supervisory practice and 
student achievement. The questions the study addressed were: What does supervision 
look like in an effective school and how do supervisors effectively work to influence 
teacher behavior that best promotes student learning? What impact does successful 
supervision have on student achievement? What can we learn from this case study that 
might inform the practice of supervision in other schools? Based on this tentative case 
study involving one school, findings indicate that supervision is purposeful, targeted, and 
central to promoting a school wide instructional program wherein student achievement is 
always at the forefront. Principal leadership is essential as is the establishment of a 
culture of teacher empowerment and collaboration. The paper concludes with some 
questions for continued study into the connection between supervision and student 
achievement. 
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Impact of Instructional Supervision on Student Achievement: 
Can We Make the Connection? 



“The purpose of educational accountability is to improve student achievement.” 

Douglas B. Reeves (2002) 



Preface 

This paper is the third and final installment in an on-going three-year study on the nature 
of instructional supervision in the New York City public schools, that have undergone 
radical organizational and curricular transformation since the mayor, Michael 
Bloomberg, took over control of the schools and centralized authority in the hands of a 
new chancellor, Joel Klein, an attorney. The first paper, entitled “High Stakes Testing, 
Standards, and the Demise of Instructional Supervision,” reported on the status of 
instructional supervision within the prescriptive standards-based environment based on 
interviews with New York City public school teachers. The research findings 
demonstrated the prevalence of directive, checklist and narrative approaches to 
supervision. Principals and their assistants were perceived by teachers as predominantly 
being involved in non-instructional duties and evaluative functions. Supervision, as a 
means to promote instructional dialogue to improve teaching in the classroom, was 
carried out by a new position called an “instructional coach” who was neither a 
supervisor nor trained in instructional supervision. Moreover, although reform efforts 
called for greater professional development (PD), data indicated that most PD workshops 
were not viewed as particularly helpful, nor were they sustained and teacher initiated. 
Little connection, if any, between supervision and professional development was reported 
by survey respondents. Respondent decried, for the most part, the emphasis on 
prescribed standards and increased testing of students. The researchers concluded that 
the standards-based environment in New York City with its emphasis on uniform 
curricula provided onerous challenges to quality instructional supervision. A major 
conclusion from this study was that instructional supervision, as best practice, was mostly 
absent in the schools in which the respondents to the surveyed worked. 



The second paper, “Usurpation or Abdication of Instructional Supervision in the New 
York City Public Schools?” continued to report on the status of instructional supervision 
under the influence of a centralized and bureaucratically managed system (i.e., top down 
initiatives) in New York City. Surveys from teachers, coaches and principals indicated 
that supervision, for the most part, remained a monitoring or inspectional task 
reminiscent of older forms of supervision. Supervision remained the responsibility 
primarily (although unofficially) of coaches who had little, if any, formal training in 
supervision. The underlying question of this study was the extent to which supervision as 
improvement of instruction had been abdicated by principals or usurped by coaches (not 
personally but as a result of bureaucratic and administrative fiat) given their charge to 
work with teachers in the classroom. Factors that precipitated such a situation were 
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explored by reports from interviews with New York City public school teachers, coaches, 
principals, and assistant principals. Findings indicated that supervision was not being 
usurped by coaches in the sense that they had purposefully intended to assume 
supervisory duties, traditionally relegated to trained principals and their assistants. Given 
the structure and nature of reform efforts in NYC, coaches played a significant role in 
instructional supervision. However, without vested authority and little or no training in 
supervision, coaches had little impact on altering teacher behavior that supports quality 
teaching needed to promote student achievement. Moreover, principals hadn’t 
intentionally abdicated their instructional supervisory function as much as had been 
overwhelmed by the enormity of the reform measures and constant struggle to keep 
“head above the water,” as one principal admitted. In another sense, however, principals 
had abdicated their responsibility by complying so readily with reform measures, many 
of which made little instructional sense. 

For the third and final study in the series, the researchers wanted to assess the impact of 
instructional supervision on student achievement. As indicated by the previous two 
studies, although instructional supervision was not given its due attention, some schools 
surveyed did, in fact, display some rather creative and comprehensive approaches to 
supervision. Examining one such school forms the basis of this study. 



Introduction 

This final paper, picking up from two studies previously reported during the last two 
AERA conferences, reports on results of an attempt to find a link between instructional 
supervision and student achievement. In other words, how can researchers assess the 
impact of supervision and professional development on student learning? Previous 
research (Eeithwood, Seashore Eouis, Anderson, & Wahlstrom, 2004) demonstrate that 
such a link is indirect (also, see Alig-Mielcarek, 2003). Still, what impact can 
supervision have on teacher classroom behavior that might in turn positively influence 
student achievement? 

The researchers decided to undertake this important assessment on a small scale by 
utilizing an in depth case study of one school where such a connection is less tenuous and 
more demonstrable than perhaps other settings. This paper presents its findings and 
draws some implications, and raises still many more questions. Data were collected 
through in depth interviews with school building administrators, coaches (still in process 
as of this writing), and teachers (still in process as of this writing). Close examination of 
standardized test scores were undertaken. An instrument is currently being developed to 
assess the effectiveness of supervision within schools matched to school performance on 
standardized tests. Data obtained from administrators and teachers were used to confirm 
or reject student achievement comparisons. Non-participant observational methods were 
also incorporated. In sum, this instrumental case study examined the nature of 
instructional supervision and its impact on student achievement. 

The following questions were addressed: 

1. What does effective supervision “look like?” 
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2. What impact do supervisors have on teachers’ in-class teaching behaviors and 
attitudes towards promoting student learning? 

3. What is the connection among instructional supervisory practices, teacher 
classroom behavior, and levels of student achievement? 

4. What can we learn about making the connection between supervision and student 
achievement? 

Raising questions for further research into the connection between supervision and 
student achievement forms the conclusion section of this paper. 



Background 

The Handbook of Research on School Supervision (Firth & Pajak, 1998), the most recent 
attempt to report on the status of the field of supervision from a research prospective, has 
little or nothing to say about supervision’s impact on student achievement. In over 1250 
pages, the only reference, cited in the index about student achievement, occurs in 
reference to coaching (Goldsberry, 1998). Summarizing research in the field, Goldsberry 
(1998) emphatically states that “Scant evidence of learners’ performance or attitude 
improvement resulting from coaching programs exists” (p. 447). He later states that 
“When student achievement measures are used, the connection between achievement and 
the contribution of coaching is necessarily tenuous” (p. 447). The absence of research on 
the impact of supervision specifically related to achievement has been lamented at many 
COPIS and AERA-SIG annual meetings and during informal conversations among 
professors of supervision. Without adequate research addressing specific ways 
supervision works to influence student achievement, supervision as a field and practice 
will continue to remain inconsequential in an era of heightened accountability that 
measures efficacy of instructional strategies by examining links to levels of student 
achievement. Notwithstanding, some methodological, technical, and even logistical 
difficulties in being able to make firm connections, scant attempts to address the issue 
have thwarted efforts of supervision to make a unique contribution under current school 
reform efforts. This paper is an attempt to add to the conversation by raising questions 
and attempting to make the connection between supervision and student achievement a 
bit less tenuous. 

Efforts to connect school, if not only instructional, leadership to student learning have 
recently been underway. One of the most recent efforts to make such connections were 
reported in a comprehensive study by Eeithwood, Seashore Eouis, Anderson, and 
Wahlstrom (2004). They posed the following questions, among others: 1) What effects 
does successful leadership have on student learning?; 2) How does successful leadership 
exercise its influence on the learning of students? The researchers found that “successful 
leadership can play a highly significant . . . role in improving student learning” (p. 3). 
Cautioning readers about the size and effects of such conclusions, the researchers drew 
the following conclusions, among several others: 
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1. Leadership is seeond only to elassroom instruetion among all sehool-related 
factors that contribute to what students learn at school. 

2. Leadership effects are usually largest here and when they are needed most. 

3. Principals, ... are being admonished to be “instructional leaders” without much 
clarity about what that means. 

Waters, Marzano, and McNulty (2003) also reported on the effects of leadership practices 
on student achievement. The researchers conducted a meta-analysis of many studies 
from the 1970s, including dissertations. After examining over 5,000 studies, the authors 
developed a framework that identified 21 leadership behaviors that significantly 
correlated with student achievement. They published their findings with prescriptions for 
practice (Marzano, Waters, & McNulty, 2005). Supervision was not explicitly addressed. 

Alig-Mielcarek (2003) hypothesized that “instructional leadership would have a “direct 
effect on student achievement (p, 120). However, the author concluded “the data did not 
support this hypothesis” (p. 120). Although “bivariate correlations . . . indicated that 
instructional leadership has a significant positive relationship with student achievement,” 
(p. 120), further studies are required such as “longitudinal” analyses. Hallinger and Heck 
(1996) explain that although principals might impact student achievement, such an 
impact might be greater among “others in the school” (p. 26). Supervision, specifically, 
is not addressed by either study. 

Williams (2003) in a dissertation titled The Relationship between Principal Response to 
Adversity and Student Achievement emphasized the importance of the principal in 
influencing student achievement through developing a school culture focused on learning 
and working to establish a collaborative learning community. The researchers used an 
ex-post facto research design to examine the relationship between an Adversity Quotient, 
a self-reporting instrument, and scores from standardized student achievement data over a 
two year period. Results indicated that students attained higher test scores in schools 
with higher Adversity Quotient principals. 

In a similar, yet more recent study, Owings, Kaplan, and Nunnery (2005) conducted a 
statewide study to “determine the relationship between principal quality as measured by 
ratings on an ISLLC standards rubric and student achievement scores over time” (p. 102). 
Student achievement levels were higher in schools with principals with higher ratings. 
The researchers concluded that principal quality was connected to student achievement. 
They caution, though, that “the relationship is correlational and not causal” but that “it is 
reasonable to believe that principals who practice and build skills in leadership for 
teaching and learning can positively impact their schools’ learning and student 
performance” (pp. 115-116). Once again, supervisory practices per se were left 
unexamined. 

Some studies that examined leadership practices of principals discussed many factors 
except their impact of student achievement. In an extensive survey study conducted by 
MetLife (2003), data gleaned from surveys of thousands of participants nationally 
indicate that principals are critical for motivating teachers and students, ensuring a safe 
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and secure school environment, communicating to parents, and other administrative 
responsibilities. However, no specific mention is made about how principals influence 
student achievement. Again, the inference is that if such an impact were indeed true, then 
at best, it was indirect. Our review of the extant literature confirms the conclusion drawn 
by Levin (2006) in reviewing the work of Firestone and Riehl (2005) that educational 
leadership “does not produce a direct effect on student learning, but is a mediating 
influence on teachers, curriculum, instruction, community, and school organization” (p. 
40). 

Reviewing the literature of principal leadership and student achievement. Brown (2007) 
first focuses on research in the 1970s and early 1980s. Highlighting A Nation at Risk 
(National Commission on Educational Excellence, 1983) and its recommendation for 
strong leadership as a means for school improvement as well as the effective schools 
research that recognized the importance of quality leadership by consistently identifying 
strong instructional leadership as instrumental in creating a positive school climate and as 
a correlate of high- achieving schools (Edmonds, 1979), Brown indicates that such efforts 
set the tone for future emphases on measures of student achievement. Yet she reiterates 
what other scholars have indicated, “that current theory and research evidence points 
toward principals affecting student achievement indirectly.” Moreover she says:, 
“Although it is difficult to demonstrate a direct link between school leadership and 
student achievement (the most tangible and publicly accepted measure of school success), 
a model of what makes a good leader is emerging.” She identifies the following factors 
or variables as most studied: instructional leadership; school culture; management; 
communication, collaboration, and community building; and vision development, risk 
taking, and change management. She cites the work of Alexander, Entwisle, and Olsen, 
2001; Andrews, and Soder, 1987; Bender Sebring, and Bryk, 2000; Eullan, 2005; 
Hallinger, Bickman, and Davis, 1996; Marzano, 2003; National Institute on Educational 
Governance, Einance, Policymaking, and Management, 1999.; Puma, Karweit, Price, 
Ricciuiti, Thompson, and Vaden-Kieman, 1997; Scheurich and Skrla, 2003, among 
others. Brown provides a table, (Waters, Marzano, & McNulty, 2003, see Table 1 
below), indicated top ten principal leadership responsibilities that have been studied in 
regards to their effects on student achievement. Although the table mentions monitoring 
and evaluation, we argue that that is not supervision. Supervision is a non-evaluative 
process in which instructional dialogue is encouraged for the purpose of engaging 
teachers to consider effective strategies to promote student learning. Therefore once 
again, no explicit mention of supervision is proffered. 
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Table 1: Top Ten Principal Leadership Responsibilities: Average r and Associated 
Practices 



Responsibility Definition Avg 

The extent to which r 
the principal . . . 



Associated N N 

Practices school studie 

s s 



Situational 

awareness 



Intellectual 

stimulation 



Change agent 



Input 



... is aware of the 
details and 
undercurrents in the 
running of the 
school and uses this 
information to 
address current and 
potential problems. 

. . . ensures that 
faculty and staff are 
aware of the most 
current theories and 
practices and makes 
the discussion of 
these a regular 
aspect of the 
school’s culture. 



... is willing to and 
actively challenges 
the status quo. 



. . . involves 
teachers in the 
design and 
implementation of 



.33 ■ Is aware of informal groups 91 

and relationships among 
teachers and staff 

■ Is aware of issues in the 
school that have not surfaced 
but could create discord 

■ Can predict what could go 
wrong from day to day 

.32 ■ Stays informed about current 321 

research and theory 
regarding effective 
schooling 

■ Continually exposes teachers 
and staff to cutting edge 
ideas about how to be 
effective 

■ Systematically engages 
teachers and staff in 
discussions about current 
research and theory 

■ Continually involves 
teachers and staff in reading 
articles and books about 
effective practices 

.30 ■ Consciously challenges the 479 

status quo 

■ Is comfortable leading 
change initiatives with 
uncertain outcomes 

■ Systematically considers 
new and better ways of 
doing things 

.30 ■ Provides opportunities for 504 

input from teachers and staff 
on all important decisions 
■ Provides opportunities for 
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8 



Culture 



Monitors/ 

evaluates 



Outreaeh 



Order 



important deeisions 
and policies 



teachers and staff to be 
involved in policy 
development 
■ Involves the school 
leadership team in decision 
making 



. . . fosters shared .29 
beliefs and a sense 
of community and 
cooperation 



. . . monitors the .28 

effectiveness of 
school practices 
and their impact on 
student learning. 



■ Promotes cooperation 709 13 

among teachers and staff 

■ Promotes a sense of well- 
being 

■ Promotes cohesion among 
teachers and staff 

■ Develops an understanding 
of purpose 

■ Develops a shared vision of 
what the school could be like 

■ Monitors and evaluates the 1071 30 

effectiveness of the 

curriculum 

■ Monitors and evaluates the 
effectiveness of instruction 

■ Monitors and evaluates the 
effectiveness of assessment 



... is an advocate or .28 
spokesperson for 
the school to all 
stakeholders. 



■ Advocates on behalf of the 
school in the community 

■ Interacts with parents in 
ways that enhance their 
support for the school 

■ Ensures that the central 
office is aware of the 
school’s accomplishments 
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. . . establishes a set .26 
of standard 
operating principles 
and procedures. 



■ Provides and enforces clear 456 
structures, rules, and 
procedures for teachers, 

staff, and students 

■ Establishes routines 
regarding the running of the 
school that teachers and staff 
understand and follow 

■ Ensures that the school is in 
compliance with district and 
state mandates 
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Resources . . . provides .26 

teachers with the 
material and 
professional 
development 
necessary for the 
successful 
execution of their 
jobs. 

Ideals/beliefs ... communicates .25 

and operates from 
strong ideals and 
beliefs about 
schooling 



■ Ensures that teachers and 570 17 

staff have necessary 

materials and equipment 

■ Ensures that teachers have 
necessary professional 
development opportunities 
that directly enhance their 
teaching 

■ Holds strong professional 526 8 

ideals and beliefs about 

schooling, teaching, and 
learning 

■ Shares ideals and beliefs 
about schooling, teaching, 
and learning with teachers, 
staff, and parents 

■ Demonstrates behaviors that 
are consistent with ideals 
and beliefs 



Waters, T., Marzano, R.J., & McNulty, B. (2003). Balanced leadership: What 30 years of 
research tells us about the effect of leadership on student achievement. Aurora, 
CO: Mid-continent Research for Education and Eearning. 



Recently, Pajak (2006) has begun a study focusing on student achievement through 
curriculum leadership as part of a larger instructional supervision effort. Results of this 
study, however, are forthcoming. Goldsberry (2006) at the same conference raised 
provocative issues about the connection between supervision and student achievement, 
but no definitive results of a study were reported. Not many other researchers are 
focusing on supervision. Consequently, the fall, 2007 COPIS conference has as its theme 
“Supervision and Student Achievement” as an attempt to catapult interest in this very 
important area for research. 

Although little, if any research exists focusing on supervision and student achievement, 
we do know that supervision as best practiced stresses work with teachers on specific 
teaching and learning strategies. Marzano, Pickering, and Pollock (2001) have elucidated 
specific research-based instructional strategies that affect student achievement. Since 
presumably supervisors work with teachers to improve instruction, if supervisors would 
work on helping teachers apply these research-based strategies, then supervisors would 
indeed have an effect of student learning, albeit indirectly. Yet, no study has been 
conducted to test such a postulate. 
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Despite the lack of mention of instructional supervision as a variable under study, we 
might be able to draw some conclusions about what possible effects supervision would 
have on student achievement from results reported by the aforementioned studies. 
Successful leadership, in general, appears to have an indirect influence on the school 
organization and thus on student learning. Moreover, research affirms that educational 
leaders who pay close attention to instructional matters at the classroom level effect 
successful teaching, and thus learning; but again, it’s an indirect influence. We still do 
not know enough about what educational leaders do to influence learning, nor the impact 
of what these leaders do versus a plethora of other variables including, among others, 
teacher preparation and certification status (Darling-Hammond, 1999; McColskey & 
Stronge, 2006), school or district culture (Hargreaves, 1995), class size (Hoxby, 2000), 
etc. As we will underscore later in the paper, supervision as an intervening variable 
needs close attention and study. Superficially and initially, though, we may conjecture 
that if it does have an impact then that influence may be indirect, and when compared to 
other influencing variables that effect may be stronger or weaker depending on still other 
variables such as school size, experience levels of the supervisor, experience levels of 
building teachers, and demographic factors of the school and students themselves. 



Some Limitations 

Although the researchers in this study attempted to clarify the connection between 
supervision and achievement, several significant limitations persist. Obviously, one case 
study, albeit incomplete as of this writing, is insufficient to make a strong connection. 
Still, the researchers chose to examine one particular school to more deeply understand 
how one school, considered successful (i.e., lauded by district administrators as 
evidenced by high achievement test scores), incorporated instructional supervision as the 
mainstay of its approach to foster student learning. The limitation is obvious; results 
cannot be generalized beyond this one particular school. Moreover, idiosyncratic 
practices vary by school. Still, the researchers were able to discern “best practices” in 
supervision as reflected in the literature and testimonies of school practitioners. 

Another limitation, centers on the reliability of data derived solely from standardized 
tests. Drawing student achievement conclusions about any school practice (e.g., teaching 
strategies, principal leadership behaviors, or supervisory practices (e.g., clinical 
supervision) must be viewed in a comprehensive manner. Limitations of tests 
themselves, renorming practices, and efforts by educators to train students in test 
sophistication strategies need consideration. Addressing some of these concerns, Meier 
and Ravitch (2006) lament excessive emphasis on standardized tests under current NCLB 
legislation because they negatively impact curriculum and teaching in schools. They 
further argue that testing gains, where reported, are suspect. They explain: 

Yet both of us are appalled by the relentless “test prep” activities that have 
displaced good instruction in far too many urban classrooms, and that narrow the 
curriculum to nothing but math and reading. We are furthermore distressed by 
unwarranted claims from many cities and states about “historic gains” that are 
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based on dumb-downed tests, even occasionally on downright dishonest scoring 
by purposeful exclusion of low-scoring students. 

This study raises cautionary notes about making definitive connections between 
supervision and achievement. However considering the scant attention of past 
scholarship in the field of supervision related to assessing impacts on student 
achievement, this paper presents one, of hopefully future attempts to better understand 
the work of supervision in schools and its impact on teacher behavior and, ultimately, on 
student learning. 



Methods and Data Sources 

The Case: PS X is located in a rapidly changing southern Brooklyn neighborhood. The 
early 20* century traditional brick building houses 755 Pre-K to 5 students. A 
previously predominantly Italian-American neighborhood, the surrounding Asian and 
Hispanic populations have increased significantly over the past 5-10 years. The present 
school population is 62% Asian, 21% Hispanic, 14% Caucasian, and 3% black. The 
percent of students eligible for free lunch increased in the years between 2003 and 2005 
from 73% to 93.7%. These statistics are significant in the analysis of student 
achievement scores. 

Ms. X was appointed to PS X in Fall 2004. She had previously been an assistant 
principal for four years at an intermediate school in the same school district. She 
received her leadership training through a district/university partnership where 
instructional and transformational leadership and a year-long full-time internship were 
the principal focuses. The former Assistant Principal received similar training, although 
not through the collaboration. She had been a coach and a staff developer and was hired 
as an assistant principal before completing the leadership program because of her 
experience, reputation, and skills as a teacher leader. 

The Study: We sought to examine the development and evolution of instructional 
supervision in this particular school and its impact on student academic achievement. We 
utilized Glanz and Behar-Horenstein’s (2005) conception of instructional supervision, 
broadly defined as a “process which utilizes a wide array of strategies, methodologies and 
approaches aimed at improving instruction and promoting educational leadership as well 
as change (p.85). We conceptualized this project as an instrumental case study (Denzin 
& Lincoln, 1988), where the primary goal is to promote understanding of an issue or 
theory through the examination of a particular case. Yin and Campbell (2002) 
distinguish the case study from other research strategies as optimal when one investigates 
a contemporary phenomenon within its real-life context, when the boundaries between 
phenomenon and context are not clearly evident; and one in which multiple sources of 
evidence are used. 

We used mixed methods as the mode of inquiry (Onwuegbuzie, 2003, 2005) with the 
intent both to capture the individual’s point of view through detailed interviewing and 
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observation, and to examine a profile of student achievement over several years. We 
took a case-based position that examined the characteristics of the particular context, and 
we sought to secure rich description. Kvale’s (1996) conception of life world interviews 
fit with our view that conversations with key school personnel would provide an 
understanding of the supervisory practices that were espoused and in use. We used semi- 
structured interviews (Kvale, 1996) in conversations with all participants, including the 
principal, assistant principal and teachers. To maintain anonymity, we identified 
participants only by whether they were administration or teachers. The authors’ 
professional relationship with this particular school permitted additional interviews, 
numerous site visits, and the collection of written information about the different 
programs. 

The qualitative methodology employed was characteristic of research as described by 
Merriam (1997) and others: we were observers, we used a small sample of interviewees, 
our interview data were not described numerically, and we used multiple examples and 
quotations in reporting our findings. As our study proceeded, we interwove data 
collection and analysis using constant comparative analysis (Strauss & Corbin, 1998) that 
allowed us to clarify where the data fit with and departed from current theoretical 
constructs about supervisory practices. 

Typically, we carried out individual, one-on-one interviews with participants, transcribed 
the interviews verbatim, and checked them for accuracy. The texts were read and reread 
a minimum of three times by two of the researchers, with the goal of identifying 
categories and recurring themes. Unstructured questions, which arose during the 
interviews and from the constant comparative analysis of other interviews, clarified the 
basis for the patterns that emerged. 

Interview questions for school personnel included: 

1 . What do you believe are the most significant factors leading to improved achievement 
in this school? 

2. What does the supervisory process look like in your school? 

3. What types of classroom supports have been provided in the implementation of the 
DOE initiatives? 

4. How has student information, such as standardized test scores, been used to support 
classroom instruction? 

5. What does professional development look like in your school? 



We employed triangulation procedures to reduce the likelihood of misinterpretation of 
data and to clarify the meaning of our interviews and observations. While multiple data 
sources (including interviews, focus groups, questionnaires, observations, and data 
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documents) were used for triangulation (Merriam, 2001; Strauss & Corbin, 1994), 
findings reported were primarily from interviews and observations. Formal and informal 
observations (including informal conversations with selected teachers at the school) were 
conducted by two of the researchers. Documents provided supplemental information, 
and included district strategic plans, policies, descriptions of district and site initiatives, 
and curriculum information. Student achievement data, obtained over the last three 
years, was obtained from the New York City Department of Education website. The 
depth and variety of the data collection procedures that we adopted supported the validity 
of the findings and conclusions. 

At the time of this writing we have not yet completed the quantitative assessment that 
included the administration of a two-part survey to all school personnel. Part one of the 
survey will assess background information about participants, including teaching and 
educational experience. The second part of the survey will consist of twenty Likert-style 
items concerning participants’ views about effective supervision, instructional leadership, 
and the relationship between supervision, classroom instruction and student learning. In 
addition, participants will be asked to respond to a checklist of different supervisory 
practices, indicating which practices had been experienced and/or utilized by them in 
their school. Focus groups have also not yet been conducted. Logistics at the school as 
well as city-wide mathematics testing precluded survey distribution and focus group 
interviews. If the researchers are able to conduct these assessments prior to the AERA 
SIG session, then findings will be reported at that time. 

Student achievement data, obtained over the last three years, were obtained from the New 
York City Department of Education website. Descriptive statistics were used to describe 
this data and to compare it to data obtained from comparable schools in the 
neighborhood. 



Findings 

Based on interviews with the building principal and assistant principal, responses to 
several of the questions that we asked are subsumed under this first overarching question 
about school achievement. 

Question 1: What do you believe are the most significant factors leading to improved 
achievement in this school? 

The principal, who had been the school administrator for the past three years, referred to 
a systematic plan that had been in place. This particular school had become an 
“America’s Choice” school seven years before. The America’s Choice (AC) School 
Design is a K- 12 comprehensive school reform model designed, by the National Center 
on Education and the Economy (NCEE), a research consortium composed of researchers 
from Harvard, U Penn, Stanford, and U of Wisconsin. The design is based on methods of 
preventing student failure by early detection, intervention, and acceleration and conforms 
to national standards. 
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The presence of a plan, however, did not ensure success. There had been a large teacher 
turnover in recent years and newcomers were not necessarily buying in to the America’s 
Choice model. Rather, several key strategies utilized to implement the plan were viewed 
as major factors in contributing to improved achievement. These included: 

1 . Building a collaborative and collegial culture. Involving school faculty in all aspects 
of capacity building was seen as the major contributor to the school’s success. Teachers 
were empowered to work with the different facets of the AC plan and to give input about 
program successes and failures. Teachers became motivated to go beyond what was 
required. In the words of the principal: 

The biggest diff is the culture and school climate which makes it collaborative 
and encourages teachers to buy in. WE also got rid of things we didn’t’ like 
with America’s Choice and that came from teachers. The culture building was 
huge. . . .They [teachers] want to do more because no one says you have to. 

The principal described “getting rid of’ facets of the America’s Choice program that 
teachers didn’t like. When asked to elaborate she described a change in school climate: 

There was a sense that the administration said what should be. We changed that to 
‘we need your input, we respect what you know and you.’ There was the attitude 
that the coaches were in charge and would go into classrooms. That’s changed 
because they are not supervisors. 



2. Building teacher supports (through creative budget planning). Multiple supports for 
teachers were created. These included the utilization of instructional coaches (AUSSIEs), 
in-house mentoring for new teachers, and designing teacher schedules to include 
additional prep time each week for professional development. 

a. Instructional supports within the classroom. According to the principal, eighty 
five percent of students in this school are Asian, and fifty percent are mandated for ESE. 
When asked specifically about the kinds of classroom supports provided for the 
implementation of DOE initiatives, the administrators discussed the programs available 
and how funding for maintaining these programs was obtained. 

We have AUSSIEs, coaches. We have Title 1 money for the AUSSIEs. I 
[principal] buy the continuation package and an ESE package from America’s 
choice. . . . We have a Chinese bilingual kindergarten class. 



b. Professional development - Professional development (PD) for NYC teachers 
was taken away with the Eeb 2005 contract changes. The extra weekly prep built into 
teachers’ schedules was used for grade-wide meetings, often to discuss DOE initiatives. 
Building in extra supports like this requires the ability to be creative and innovative with 
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school budgets. This particular principal was able to find funding to hire an extra cluster 
teacher which freed up teacher scheduling to allow for this additional weekly prep time. 

The assistant principal discussed the advantages of building PD into the school day: 



When teachers stay after school you have to pay them and you don’t always get to 
deliver message to all parties that need it. So building it into school day is more 
cost effective and you can reach a full audience. 

The principal noted how teachers resented additional prep time until it was built into the 
school day. She spoke about all kinds of professional development opportunities that are 
made available for teachers: 

We have a lot of professional development - Carl Anderson - guru of assessing 
students through conferencing and writing. He goes into classrooms. We avail 
ourselves of every professional development opportunity. We have people come 
in, send teachers out, have coaches, the assistant principal, myself in classroom 
every day. 

The principal, when questioned about structure of professional development in the 
school, described a multi-layered system of monthly administrative meetings, monthly 
grade conferences to assess student work and weekly curriculum meetings by grade. 
Teachers are offered half-days to attend enrichment workshops such as appearances by 
leading authors and literacy authors. 

Providing resources - The administrators discussed the availability for students and 
teachers of all the resources that are requested. Funds are often obtained through grant 
writing, and often teachers write grants for the funding that they need. The principal 
explained: 

We’re big on resources. We get everyone whatever they want. Title 1, Title 3, we 
do a lot of grant writing - teachers write grants, so we use money wisely and 
make sure kids get what they need and teachers too. 

The previous assistant principal’s response to acquiring needed funds: 

You look at the budget and cut back on something else- like supplies. Funding for 
teachers is essential. If the teachers aren’t on board your program won’t succeed 
without teacher buy-in. 

The school’s commitment to and funding of arts projects is accomplished through the 
ability of the administration to provide needed resources, described as follows: 

We are committed to the arts. We have a visual and vocal and dance and theater 
art cluster. So you fund that through what? Through tax levy and through Title 1 
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because it’s really professional development money for that one. We have a band 
teacher funded thru project Arts, several arts related after school clubs, 
instrumental - violin - that’s a person who comes in and works with Kindergarten 
children, (interview with principal) 

The assistant principal described how arts enrichment is built into the teachers’ 
contractual day, providing enrichment for students who don’t need tutoring: 

I know that M [principal] had built a lot of arts enrichment into her 371/2 minutes, 
students who aren’t mandated to attend for tutorial can come in for club time. She 
has a staff that is very open to this over there. For example, she has her chorus 
coming in the afternoon. They are limited to 10 kids per teacher - so she’ll have 
one music teacher and 3 others with an interest in music. It’s the same thing with 
art - there is art teacher and another teacher who is interested in art. She also has 
sign language, a percussion group... a dance club and a theater club. Those are 
built into her 371/2 minutes. So she’s providing a lot for those students but 
building it intothose minutes which are part of the teacher contract. 

3. Building capacity. Teachers are responsible for the professional development and 
mentoring of their colleagues. The principal describes this as follows: 

We are building capacity - letting teachers do professional development and 
training. Our new teachers have an in-house mentor in addition to the coaches. So 
they can watch each other teach, they don’t have to report to me - it presents a 
risk-free environment for growth. They know their colleagues aren’t coming back 
to report to me - unlike the coach. 

4. Administrators as instructional leaders. School administrators are involved in all 
aspects of instruction. The principal and AP spend 2-3 hours a day in classrooms. “I’m in 
the classroom all the time,” is the principal’s description of her day. When asked about 
teachers’ reactions to this kind of informal observation, the principal responded: 

they don’t even notice it - I’m sitting with kids. I’ll go in and find the kid with 
the math problem. So they don’t think of it as evaluation, it’s natural for me to be 
in the classrooms. Sometimes we’ll talk about something I noticed, a concern. 
They know I never walk through the building on Friday afternoon. 

5. The use of student information to support classroom instruction. Responding to a 
question concerning how student information was used to support classroom instruction, 
the administrators provided a detailed description of the “monitoring for results” system 
that is in place in the school. Teachers assess students four times a year and coaches 
compile the results. Staff then meets in “people personnel teams” (PPT) that include the 
principal, assistant principal, teacher, ESL teachers, providers such as speech and 
resource room. These teams plan a course of action for each child. While the monitoring 
for results system had already been in place, academic intervention has been greatly 
expanded. 
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Expert use of resources was again noted in the principal’s description of the expanded 
academic intervention services, including the hiring of extra personnel: 

I put in a third Reading Recovery teacher and the Leveled Literacy Intervention 
(LLI) teachers. We have a floating teacher - who comes in and does intervention 
with kids. She does it all day long, 5 days a week. I was given her but I didn’t put 
her in the class. She’s not someone I would want in the class all day long but she 
does nicely one on one with kids so that’s what we use her for. 

When asked about support for parents, the administrators listed a variety of parent 
programs including ESL, workshops on assorted topics, visiting social workers and 
psychologists, ballroom dancing, parents as arts partners, and parent movie nights. While 
there is a talented parent coordinator who works on these programs, getting all parents 
involved is viewed as a challenge. The principal described a lack of involvement with 
some of the parents who work long hours and may not deem it essential that they 
participate in their children’s school life. 

Observations made by the researchers confirm the reports above. A detailed description 
of these observational data is forthcoming. 

Question 2: What does the supervisory process look like in your school? 

The supervisory process was described as informal and collaborative. The administrators 
spend a good part of their day in classrooms and in informal meetings with teachers: 

The AP and myself are in building all day long. A lot of teacher collaboration. I 
meet with them to discuss their conference notes that they take on kids, and their 
assessments. It gives an insight into how much they know about what they are 
getting from kids. So I spend a lot of time doing that. 

Asked about formal observations, the principal responded: 

There is no such thing as a formal observation, no one is wearing a tuxedo and if 
you want to go in and observe every day you go in. (Attributed to Vincent Grippo, 
former school administrator in the district). 

Teachers who need support get extra time to work with coaches and/or their fellow 
teachers. When asked if she gives teachers unsatisfactory ratings the principal responded 
as follows: 

I haven’t had any [unsatisfactory ratings] here. I did have them at [school X]. I’ve 
had teachers with not great formal observations, and could have been a U, but met 
with them and supported them and come back to do another formal observation. 
I’ll support. There is no teacher that I’ve hired personally that would rate a U. 
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The principal concluded the interview reiterating the importance of school tone: 



Kathy, my AP, said this morning that people here have it good. I think you can 
supervise and I think it’s the culture. They know I’m not out to get them but if 
you’re not doing your job I’m going to call you on it. If it’s not good for kids it’s 
not happening here. 

Observations made by the researchers confirm the reports above. A detailed description 
of these observational data is forthcoming. 



Findings based on achievement data follows: 

Table 1. New York City Test Results in English Language Arts (ELA) 

Percent of students meeting the standards in all tested grades (performance levels 3 & 4). 



Year 


School X 


Similar Schools* 


All City Schools 


2003 


69.6 


37.1 


49.1 


2004 


73.6 


36.2 


48.3 


2005 


80.9 


49.2 


60.9 



Note: Tested grades include grades 3 & 5. 

*Similar Schools are those defined as having a similar percent of students eligible for the 
Free Lunch program, a similar percent of tested Special Education students, and a similar 
percent of English Eanguage Eearners. 

Eevel 4: Students exceed the learning standards for English language arts. Their 
performance shows a superior understanding of written and oral text. 

Eevel 3: Students meet the learning standards. Their performance shows a thorough 
understanding of written and oral text. 



Table 2. New York City Test Results in English Language Arts (ELA) 

Percent of students far below the standards in all tested grades (performance level 1). 



Year 


School X 


Similar Schools 


All City Schools 


2003 


2.8 


19.5 


14.1 


2004 


3.3 


19.1 


13.8 


2005 


1.1 


13.2 


9.3 



Note: Tested grades include grades 3 & 5. 



Eevel 1: Students do not meet the learning standards for English language arts. Their 
performance shows minimal understanding of written and oral text. 
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Table 3. New York City Test Results in Mathematics 

Percent of students meeting the standards in all tested grades (performance levels 3 & 4). 



Year 


School X 


Similar Schools 


All City Schools 


2003 


73.4 


41.2 


52.3 


2004 


77.2 


44.1 


55.1 


2005 


88.2 


55.1 


65.1 



Note: Tested grades include grades 3 & 5. 



Level 4: Students exceed the learning standards for mathematics. Their performance 
shows a superior understanding of key math ideas. 

Level 3: Students meet the learning standards. Their performance shows a thorough 
understanding of key math ideas. 



Table 4. New York City Test Results in Mathematics 

Percent of students far below the standards in all tested grades (performance level 1). 



Year 


School X 


Similar Schools 


All City Schools 


2003 


5.7 


23.4 


17.5 


2004 


4.1 


21.6 


15.7 


2005 


3.2 


16.2 


11.3 



Note: Tested grades include grades 3 & 5. 



Level 1: Students do not meet the learning standards for mathematics. Their performance 
shows minimal understanding of key math ideas. 



Note: Grade 4 students take the New York State ELA (English language arts) and 
mathematics. These test results show three year increases in the number of students who 
meet or exceed performance levels (levels 3 & 4). Performance over three years (2003- 
2005) on the state EEA increased slightly from 74.2% to 78.6%, and compared quite 
favorably to the performance of similar schools, where the percent of students scoring at 
levels 3 & 4 rose from 42/2% to 49.9% over three years. Performance over three years 
(2003-2005) on the state mathematics exam showed large increases from 78.1% to 
90.2%, and compared quite favorably to the performance of similar schools, where the 
percent of students scoring at levels 3 & 4 rose from 58.1% to 71.3% over three years. 



Discussion 
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What struck the authors in reading the literature about connecting leadership/supervision 
of instruction to student outcomes were two blatant omissions: 1) the absence of specific 
references to supervision of instruction (as noted in the Background section above), and 
2) the lack of resolution of the dilemma that Leithwood et al. (2004) refer to: Principals, . 

. . are being admonished to be “instructional leaders” without much clarity about what 
that means.” This dilemma arises in part from the difficulty in distinguishing 
instructional leadership from supervision of instruction. In fact, the Special Interest 
Group (SIG) under which this paper falls recently changed its name from “Instructional 
Supervision” to “Supervision and Instructional Leadership” because of this quandary. In 
examining one school that had made significant increases in the State standardized tests 
since a new principal and assistant principal were appointed three years ago, we realized 
that a clear expanded definition of “supervision of instruction” could encompass most of 
the strategies and practices that we found were the link between supervision, teaching, 
and learning, and student achievement based on achievement tests and a descriptive 
evaluation of curriculum and practices. 

Therefore, we have expanded on the following definition of supervision of instruction 
included in our methodology, “that process which utilizes a wide array of strategies, 
methodologies and approaches aimed at improving instruction and promoting educational 
leadership as well as change” (Glanz & Behar-Horenstein, 2000, p. 85). The collegial 
model that Glickman, Gordon, and Ross-Gordon (2006, p. 6) describe provides a more 
specific link between leadership, supervision, and teacher growth that can impact on 
outcomes: 

1. A collegial rather than a hierarchical relationship between teachers and formally 
designated supervisors. 

2. Supervision as the province of teachers as well as formally designated supervisors. 

3. A focus on teacher growth rather than teacher compliance. 

4. Facilitation of teachers collaborating with each other in instructional improvement 
efforts. 

5. Teacher involvement in ongoing reflective inquiry (Gordon, 1997, p. 1 16). 



Our interviews and observations revealed the actualization of these five tenets through 
specific beliefs, strategies, and practices that resulted in improved classroom instruction 
and consequently higher student outcomes. We believe that the description of what 
actually occurred in school X contributes towards an understanding of that black box of 
what it means to be an “instructional leader,” or more precisely a collegial supervisor of 
instruction. 

The field of instructional supervision has been struggling to find its role in contemporary 
leadership because the search for the understanding of what effective instructional 
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leadership is has disregarded the fact that collegial supervision of instruction may be the 
key underlying factor in improved classroom instruction and student learning. Our 
findings enable us to connect leadership to improved student outcomes in two significant 
ways. The two significant concepts behind certain strategies and approaches the 
administrators in this school have implemented that fall under the umbrella of collegial 
supervision are: 

1) The role of the teacher in developing and modifying programs and practices to 
improve teaching and learning. The teachers made decisions about the use of the 
“America’s Choice” support. For example, they reduced the “Monitoring for Results” 
student tracking program from four to three times a year because they felt that it was 
taking up too much time and didn’t coincide with their report cards. The teachers are 
responsible for much of the professional development within the school and turnkey in 
the school what they learn in the off-site professional development that the principal 
strongly supports. 

2) The leader’s ability to offer supplementary support for teachers and students beyond 
the support programs already in place, to allow them to be the most effective possible i.e., 
adding an in-school mentor to the Department of Education mentor and the school coach 
to ensure sufficient support for new teachers, freeing teachers up for intervisitations, 
adding a second year of individualized reading support for at-risk children, and adding an 
extra prep period for common planning. 

The following areas are less directly related to supervision of instruction, but result from 
strong supervisory skills. 

Many programs were already being implemented when the current principal was 
appointed to school X. “America’s Choice” had been in place for four years; coaches 
existed for two years, and additional supports such as AUSSIES (Australian balanced 
literacy consultants) and Reading Recovery (a first grade individualized support program) 
were ongoing. As mentioned in a previous paper (Shulman, Sullivan, & Glanz, 2006), 
the literature has confirmed the importance for improvement of continuity of efforts over 
a 5-10 year period. This principal chose to continue these efforts and intensify them 
where appropriate to insure increased effectiveness. 

Another crucial area that emerged indirectly related to supervision of instruction was 
selection and utilization of staff. The principal revealed that she has not had to give any 
unsatisfactory ratings because of her careful hiring process. Several consequences result 
from her effective hiring: 1) time can be spent collaboratively nurturing and supporting 
staff because of the lack of need for monitoring; 2) strong teachers can be teacher leaders 
involved in all the collaborative supervision and support processes in place in the school; 

3) the culture becomes a collaborative one because no one fears being fired; thus, the ease 
in building a community of learners. In addition, the principal’s strong problem-solving 
skills has enabled her to resolve staffing problems over which she had limited control, 
such as the excessed teacher that she was obligated to hire from a school that was being 
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reorganized. She made the decision to use the teacher to work one-on-one with students 
because she felt that she was not strong enough to work with an entire class. 



With respect to standardized achievement test scores, the citywide test score gains are 
significant for a few reasons: 1) the fact that the already high test scores continued to 
increase (a more difficult feat than raising low test scores) over the last two years can be 
attributed to continued and improved leadership. We believe that a seamless and 
effective transition in leadership and, in addition, practices, strategies, and leadership 
styles that promoted collaboration and improved instruction resulted in a continued 
growth in achievement. Furthermore, the increase in children eligible for reduced and 
free lunch from 73.1% to 93.7% during this period (2003-2005) and the increase in 
English limited proficient students from 173 to 268 out of 755 during these years make 
these increases even more significant. Although it is not possible to make a direct link 
between the leadership practices and the scores, the supervisory beliefs, strategies, and 
practices we have described are the principal changes that took place in this school over 
the last two and a half years. 

What can we learn from this case study that might inform the practice of supervision in 
other schools? Conclusions are tentative because all data are not yet in, and caution is in 
order not to extrapolate too much from one case study. Nevertheless, a direction seems 
clear. In observations conducted in this school and conversations with administrators and 
teachers, it is clear that instructional supervision plays a central role in promoting student 
achievement. Supervision is seen as critical for enhancing teacher growth. Supervision, 
in this school, is all encompassing from building a culture of reflection, collaboration, 
and improvement to encouraging leadership at all levels to offering faculty flexible and 
differentiated professional development on specific teaching strategies aimed to promote 
learning. Supervision, then, in this school appears purposeful, targeted, and central to 
school wide instructional initiatives. 



Conclusion, or Just a Beginning? 

Making the connection between supervision and student achievement has been elusive 
and tenuous (see, e.g., Witziers, Bosker, & Kruger, 2003). We have asked ourselves for 
many years, “How do we know that supervision makes a difference in terms of 
promoting student achievement?” This study tackled that question by offering an in 
depth case study of one school that may shed light on avenues for future research. 
Although we don’t think that this study provided definitive answers to that question 
(especially considering the fact that all the data are not yet it from this particular study), 
we have raised substantive and methodological questions that may form the basis for 
future research into examining how, in fact, instructional supervision works to influence 
teacher in-classroom behavior and attitudes toward student learning that, in turn, may 
affect student achievement levels. 

The following questions are posed to prompt further research possibilities: 
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1. How is supervision carried out in different schools? 

2. Does supervision at elementary versus middle school levels differ? 

3. Who is performing supervisory functions in these schools? 

4. Are supervisory practices collaborative and non-evaluative or are they 
reminiscent of past bureaucratic practices? 

5. How is successful supervision hindered by organizational constraints? 

6. What part does supervision play in the overall attempt to promote student 
achievement? 

7. How is supervision different, if at all, from professional development? 

8. What is the relationship between instructional leadership and supervision? 

9. How do school leaders including teachers view instructional leadership and 
supervision? 

10. To what extent does supervision of instruction positively affect teacher behavior? 

1 1 . How does teacher behavior in the classroom translate into student learning? 
(teacher “behavior” must be operationalized) 

12. What is the relationship between supervision and student achievement? 
(correlational studies) 

13. What impact does supervision have on student achievement? (quasi-experimental 
studies) 

14. How does supervision compare with other school and non-school factors leading 
to positive student achievement outcomes? 

15. What types of quantitative studies are feasible within the confines of a school 
building or district to assess the impact of supervision on achievement? 

16. What types of qualitative studies are feasible within the confines of a school 
building or district to assess the impact of supervision on achievement? 

17. Would longitudinal studies support a direct or indirect effect on student 
achievement? 

18. Which principal supervisory behaviors have the greatest effect on student 
achievement? 

19. Which assistant principal supervisory behaviors have the greatest effect on 
student achievement? 

20. How can gains in achievement be attributed to supervisory behavior versus a 
plethora of other intervening variables? 

Most recently, Fullan, Hill, and Crevola (2006) have underscored the critical importance 
of high-quality instruction and its systematic delivery as most necessary in order to 
ensure “continuous improvement and ongoing academic success.” Our work in 
supervision can play a major role in such an effort. What we need is a concerted 
systematic approach to assessing the impact our work with teachers has in terms of 
promoting student achievement. We call on fellow supervision scholars to focus their 
research efforts over the next decade on direct measures of student achievement. As a 
community of scholars, we have not attended to such analyses. If we fail to do so, our 
work will remain inconsequential and, more importantly, we will not be able to 
contribute to the important work of helping teachers successfully promote student 



24 




learning, especially in light of the current and ongoing accountability exigencies we face 
on a daily basis. 
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